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It is fdt that an important source of information in engineering edocahon is otte 
n overlooked - the differential impact of the learning process on each stu^t. 
Curr^y. the only measure of this sort is the final grade. research mvdved (l) 
developinq techniques for assessing various asp^ts of a students class 
perforSia^, and (2) examination of the usefuln^s of the 
averaqe. Two different attacks on the problem were 

corrditional study of grading oractices, and (2) developm^t of both strate 9 es and 
devices for observing and evaluating engineering courses. tUHt 
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Sumnary 



Background 

The goals of engineering education have been examined from time to 
time and are currently the subject of an extensive national study. While 
meaningful discussion is being generated by the studies currently under- 
way, there is an essential source of information which is continually over- 
looked or measured only Indirectly. This information could be obtained by 
a detailed examination of the differential Impact of the learning process 
for each student; the focus of attention should be on the identification 
of the variables which determine this Impact. At present, the only measure 
of differential learning we have is a final grade. There is a wide variety 
of evidence that grades often hide more than they reveal. 

Objectives 

1. The Intent of this research is to develop techniques for assessing 
various aspects of a student's performance in specific classes. Psycholo- 
gists for the last half century have developed both a theory and a method- 
ology for the objective evaluation of various aspects of human behavior. 

The fruits of this past research have in general not Influenced the be- 
havior of most teachers. 

2. In order to demonstrate the need for paying more attention to a wide 
range of student behavior, the utility of the traditional cumulative grade 

point average is examined. 

Procedure 

Two different strategies were employed in this study. The first was 
a multivariate correlational study of grading practices. The second strategy 
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Involved the use of an observer in a wide variety of engineering classes and 
the deve lopmeiit of e veriety of essescment devices. 

A. Multivariate Correlational Study of Grades 

Two classes from the School of Engineering were chosen for this study. 
One class consisted of all those students who entered in 1961 and graduated 
in 1965. The other class is made up of all those students who entered in 
1963 and were still in engineering as of May 1966. 

For all these students we had available their scores on a wide variety 
of psychological scales. They had taken these tests as part of a routine 
battery of tests administered to all entering freshmen. 

For all men in both these classes we had recorded all their individual 
grades in all courses they had taken. These grades were then grouped accord- 
ing to principles of classification based on similarity of course content. 

We were then able to compute correlations between all combinations of grade 
classifications and all other measures of psychological traits. 

B. Techniques for Evaluating the Learning Process in Engineering Education 
Three separate courses— Dynamics, Design, and Introduction to Electri- 
cal Engineering— were chosen for study. These courses represented a broad 
range of course structures and course content. The primary Intent of this 
research was to observe and record many facets of the student's behavior 
which had a bearing, on his learning. Two facets of this behavior were 
given special emphasis: the student's general problem-solving strategy 

and the patterns of communication between student and teacher. 

Results and Conclusions 

The main report Is divided into six sections and we will report the 



3 



results and conclusions from each section. Then a brief restatement of the 
major conclusions vill follow. 

Section l-'6rading Practices and the Student's Motivation to Learn 

An assumption underlying the use of the cumulative grade point average 
is that it is an accurate reflection of the student's academic performance. 
The ubiquitous use of this measure requires that we be very clear about 
what it does and does not reflect . The undergraduate curriculum in engi* 
neering is made up of many diverse segments, and the evidence presented in 
this section suggests that mastery of one segment is no assurance of mastery 
in another. This point does not deny the significant positive relationship 
which does exist between the different sectors. Instead, the intent is to 
focus on the large part of the variance which is still unaccounted for. 

This focus can lead to examining some aspects of the teaching- learning 
process from a new perspective. The evidence suggests that a student does 
not work at a uniform level of performance in all courses. Perhaps this is 
a reality of human nature which we do not openly admit. Instead, each pro- 
fessor approaches his course with the view that he is presenting his materi- 
al to optimally motivated students. This automatically builds in a conflict 
between the instructor and at least some of his students. The student has 
the choice of meeting the expectations of the instructor or being penalized 
by a low grade. 

Perhaps we should work on the problem of how to create optimal motiva- 
tion for all students in all courses. Some of the problems inherent in this 
approach will be examined in other sections of this report. At this point 
we are particularly interested in. examining some, aspects, of the Stanford 
students' experience with the existing curriculum. 
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We have seen that aptitude is not significantly related to perfor- 
mance. In part, this can be understood to be a result of the high but 
narrow range of talent present among the Stanford students. Even though 
the narrow range of talent will reduce the possible correlation, it is 
also a fact that a student's aptitude has become less loqportant in deter- 
^1® current academic success than his previous academic success In 
high school. 

What are the consequences of a student's finding that the skills 
upon which he has previously relied for success are no longer so trust- 
worthy? We can come up with several lines of thinking which may provide 
some answers. 

Some students will find this new situation very frustrating. They 
will* therefore, respond in accord with the methods they have previously 
developed to handle frustration. We can see this operating in our finding 
that students with a well-developed sense of responsibility achieve higher 
grades than students with less responsibility. On the surface this makes 
a good deal of sense. 

In the School of Engineering we would hope that such personality 
characteristics as "responsibility" would be related to performance. As a 
professional engineer, responsibility will be a key ingredient In later suc- 
cess. When we look more carefully, however, at the meaning of the scale we 
find that there is an optimum level of responsibility. Beyond that level, 
the person would no longer be considered responsible In the positive sense 
of the term. Instead, we would begin to think in terms of someone who is 
over-cautious, unimaginative, and narrow in his ability to view problems. 

We have a little further support for this line of speculation when we 
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find that there is a negative relationship between academic performance 
and measures of self -acceptance and comfort in dealing with others • An- 
other strand of evidence comes from the general lack of relationship be- 
tween the Guilford measures of creativity and most academic areas. 

We have been suggesting that some students do well because they grind 
away at all courses , not out of genuine interest but rather through a be- 
lief in the value of perseverence . Other students make the conscious de- 
cision that they will work hard in some courses because of the Intrinsic 
worth to them and take the consequences of not working hard in other courses. 

The curriculum and the associated grading practices as they are now 
conceived and put into practice have certain undesirable consequences. The 
pressure to do well forces some students either to deny or not develop 
their own intrinsic Interests. Instead, they choose to meet the demands of 
an unrelenting faculty for work which often seems devoid of significance. 

In addition to learning the content of the courses, these students are also 

trained to be obedient and uncritical. 

Other students consciously decide or are unconsciously forced to be 
more selective in their performance. While it is true that these students 
are being forced to make mature decisions about how they will use their time, 
it is also true that they are being punished for exercising this judgment. 

There is, of course, still another group of students who do extremely 
well in nearly everything because of a combination of high ability and strong 
intrinsic interests. While in the minority, there are enough of these to 
justify maintaining the status quo. Viewed from one perspective, these stu- 
dents protect the faculty from coming to grips with the reality of how most 



students function. 
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The cumulative grade point average has become a barrier to understand- 
ing the academic behavior of students. Our intent is to get beyond this bar- 
rier by assessing what actually goes on in the classroom. 

Section 2— Toward the Development of an Ideal Problem-Solving Strategy 

The problem-solving behavior of the students was observed periodically 
throughout the school term. They volunteered to work individually on their 
homework problems, which constituted new material, in the presence of the 
researcher. While they attempted to solve the problems, they were required 
to verbalize as freely as possible what they were doing. In addition, the 
observer asked questions when clarification was needed. The following prob- 
lem-solving strategy evolved. No single student incorporated all the steps; 
that is, the listing is an amalgamation composed of procedures contributed by 
several students. The steps clearly are not independent; successful handling 
of later steps usually requires completion of those preceding. 

1. Read the pertinent section of the text thoroughly. Beginning at 
this point, knowledge of— or at least acquaintance with— preceding material 

is presumed. Usually, for each problem there were two to five pages that were 
directly related to its solution. This material had to be understood if fol- 
lowing steps were to be tackled with any substantial success. Often, more 
than one reading was necessary; "skim" reading was useless. 

2. Work through the sample problems. Texts usually contain illustra- 
tive problems helpful in conveying understanding of the content and in solv- 
ing the problems at the end of the chapter. However, thorough working- through 
of sample problems was--in the short run, at least— time-consuming and often 

neglected. 
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3. Carefully read and define the problem. This was almost invariably a 
major stumbling block. Problems often appeared vague , especially if the new 
material was not fully understood, as was usually the case. The problem typi- 
cally required several re-readings at various stages in its solution. 

4. Draw a diagram of the problem, indicating the direction of forces. 

A correct physical impression, being able to visualize what was taking place, 
was a valuable aid to solution. 

5. Know or look up appropriate formulas. Usually it was necessary to 
derive additional formulas by working from general to specific equations. 

6. If possible, arrive at an intuitive answer. This was labeled by 
one student as the "ballpark approach"; that is, attempting to get a quanti- 
tative and/or qualitative idea of the problem that was, at least, in the 
right ballpark. 

7. Try to apply the techniques of a sample problem. This was charac- 
teristically only partially successful, as the methods illustrated in the 
examples often did not directly generalize to the problem at hand. 

8. Continually check the work as it proceeds. The time-consuming and 
demoralizing effect that an error in calculation or algebra could institute 
made this a crucial concern. 

9. See how it comes out. 

Following these steps did not guarantee success. When the correct solu- 
tion did not appear, or a point of blocking occurred, students began to re- 
trace their steps. Particular emphasis was placed on a re-reading and re- 
defining of the problem. Often an accurate diagram, indicating understanding 
of the forces involved, seemed essential. Therefor-e, if the student was stumped, 
he examined and if necessary re-drew his diagram. After starting virtually from 
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scratch, the problem was re-worked. 

If an appropriate solution still was not obtained, the students charac- 
teristically fell back on another hierarchy of responses. All of them made 
use of break periods, ranging In time from five to thirty minutes. During 
their breaks, the students tried to engage In activities totally removed from 
study; for example, playing a quick game of basketball. Returning fresh to 
the problem, they were often able to figure It out. Occasionally, sleeping 
on a problem seemed to make the solution easier In the morning. 

Many students collaborated with other class members. Trading of solu- 
tions, joint study "dates," and other forms of mutual assistance were common, 
and hopeful, at least In the short run. The lone wolf was at a distinct dis- 
advantage and his scores on the problem sets were usually slightly lower than 
those of students who collaborated. 

Finally, If all else failed, the student would turn to the professor 
for assistance. A few minutes at the beginning of each class were devoted to 
questions on the assigned problems and were usually sufficient to clear up Im- 
mediate difficulties. 

There are several observations which come out of this first encounter 
with the learning process In engineering education. They are based on working 
with the students on their problem-solving and also observing the general ap- 
proach of the Instructor. There seemed to be a great emphasis on how to do a 
given problem rather than focusing on those elements of the problem which might 
be generalized to a whole class of problems. This led to students copying 
everything the Instructor put on the blackboard, but did not necessarily en- 
courage them to think. 

This emphasis on getting the job done Is certainly a. valuable part of the 
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engineering profession. It may not, however, be the most appropriate em- 
phasis in the training of engineers. 

Section 3 — Report on a Course in Engineering Design 

One purpose of the survey of Engineering 9 was to determine the ef- 
fectiveness of a questionnaire in opening channels of communication be- 
tween the students and instructors. The degree of openness in response and 
the enthusiastic participation of both students and instructor indicates the 
potential usefulness of the technique. A questionnaire has several advan- 
tages from the students* viewpoint: it is fast, yet permits open-ended com- 

ments; it is anonymous (or could be); and it does not demand face-to-face 
confrontation with a possibly threatening professor. For the instructor, 
such an instrument is easy to prepare and administer , provides an overview 
of the class's progress and difficulties, and may supply some helpful teach- 
ing advice. To the benefit of student and instructor, a questionnaire may 
serve as an "introductory** mechanism, facilitating personal contact. Several 
students, for example, expressed surprise, relief, and pleasure that their 
opinions were being taken into account by the instructor • 

If the students %dio are experiencing difficulties in any course are to 
be helped, they must be convinced of the instructor's availability. Perhaps 
the inq>lementation of weekly or biweekly requests for the students to evalu- 
ate the course and their progress is a step in the right direction. 

Concentrating on the students alone, however, does not seem sufficient. 
The instructors must feel that such contact is useful and rewarding and, fur- 
thermore, must be comfortable in what may be a relatively new role. An in- 
structor may find it necessary, for example, to act not merely as an importer 
of knowledge, but as a counselor as well. It is even conceivable that an 
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instructor and a student, by working together, may come to the conclusion 
that a student should leave engineering. For a student who is in the pro- 
cess of making such a decision, the support of an understanding faculty mem- 
ber can be extremely helpful. Similarly, for the student who decides to per- 
severe, the backing of an interested professor may make the difference between 
success and failure. 

Returning to the discussion of Engineering 9, the data make possible 
some tentative conclusions and suggestions. Within the context of a course 
which teaches the tools of descriptive geometry, it is very difficult to 
"inculcate the philosophy and methodology of design." A great many students 
perceive the course in terms of drawing, rather than design. Such an inter- 
pretation is not without Justification, as a good deal of time is focused on 
the techniques of drawing. Elements of design, conceived in terms mentioned 
above, are present but not to the degree hoped for by the instructors* 

Students in a course such as this are confronted with a confusing set 
of goals. Mastery of the technical skills of drawing is set up as one re- 
quirement. Yet many students see this as conflicting with some of the broader 
issues that are touched on in the course. Inadvertently the student who is 
more task-minded and meets the explicit requirements of the course will be 
more highly rewarded than the student who wants more challenging problems and 
thus remains unmotivated. 

The implementation of a system of written feedback to the instructor per- 
mitted some expression of student attitudes. This is a valuable first step but 
students also need to have these attitudes recognized in terms of specific 
changes in course content . The issue of change in course content is both com- 
plex and important. The needs of the student cannot adequately be met by 
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considering only one course but require an evaluation of the intended goals of 
the entire curriculum* This is beyond the scope of this study, but some of our 
conclusions will have a bearing on this topic* 

Section 4— Differential Effectiveness of Small Problem-Solving Sessions 

Students were asked to describe the quality of their problem-solving 
sessions on an open-ended questionnaire* They were also asked to race their 
instructors on a standard semantic form of the semantic differential* There 
were sharp and consistent differences between three separate classes* There 
was a strong positive relationship between a student's attitude toward the 
class and his attitude toward his Instructor* However, there was not a signifi- 
cant difference in the grades achieved by the students in the different classes* 

There is usually a plea for having excellent Instructors in the under- 
graduate curriculum* There is also some evidence that students can agree about 
the dimensions which characterize a good teacher. Students state that they 
enjoy the course more when they have good instructors* Do they also work more 
effectively and learn more? This is clearly the Important question and no 
straightforward answer is available. 

Part of the reason we don't know is the result of the very issues we are 
discussing in this report. Too often teaching is thought of only in terms of 
%diat the teacher does while actually teaching* If evaluatlcn is not considered 
a vital element in the teaching process, then there are at least two serious 
consequences. Not enough thought is given to what is being measured and the 
students' reactions to the measurement process are often disregarded* Learn- 
ing and getting good grades are too often dlstinc- processes because teaching 
and evaluation are often separately conceived* 
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Section 5 — Responses to a Problem- Orientation Questionnaire 

This initial instrument attenq>ts to measure some attitudes and behaviors 
of engineering students related to the solution of assigned problems. The 
items are aimed at eliciting the students' reactions to problem sessions and 
session instructors, the problems themselves, and problem-solving methodology. 

The questionnaire was administered to a sample of 40 students from a 
course in introductory electrical engineering. Subjects were predominantly 
juniors and seniors; no freshmen were taking the course. There were repre- 
sentatives from the electrical, mechanical, industrial, and civil disciplines 
of engineering. The questionnaire was included among other measurements of 
student attitudes. 

A majority of the students considered the problem sessions, idiich were 
conducted by the lecturer and two assistants, all Ph.D.'s, to be of some use. 
Almost unanimously, however, the students thought that the success of a pro- 
blem- session depends largely on the instructor. It is worth noting that 
students in the session taught by the lecturer, who was rated as an excel- 
lent teacher, had consistently better impressions than those instructed by the 
teaching assistants. 

Half the students wish, for example, that grades were not so dependent 
upon their ability to work problems, and 70 per cent admit that they sometimes 
get sick and tired of doing nothing but problems. Yet the great majority are 
responsible about getting assigned problems done on time. It should be noted, 
however, that hardly anyone does additional problems for self-edification. 
Apparently, the interest in solving problems is satiated for most students by 
assigned work, and spare time is spent on other activities. 

The inq>ortance of problem-solving for examination purposes is fully 



recognized, although not with a great deal of relish by many. Almost all 
students review problems before a test. On tests \diere speed of problem- 
solving is a major factor, three-quarters of the students harbor feelings 
of resentment, usually expressed in terms of "If I had had more time I could 
have done much better." Clearly, examinations which enqphasize speed alienate 
a good many students, and perhaps their utility as determinants of grades 
should be reconsidered. 

Most students share a common problem-solving behavior pattern. Al- 
though study breaks are widely employed, persistence appears to be a typical 
characteristic. Inability to solve an assigned problem results in real frus- 
tration for half the students, but hardly any lose sleep over it. Working 
through the sanqple problems, a highly recommended technique, is adopted by 
very few, primarily, it is claimed by the students, because of time pressure. 
Probably the only way to encourage students to do the sample problems would 
be to prove their time-saving efficacy. The almost unanimous belief that a 
general problem-solving strategy exists points up the necessity for the de- 
velopment of and instruction in an optimal approach to problems. 

Section 6--Some Innovations in Teaching a Section in Electrical Engineering 

In an attenqpt to investigate the effectiveness of some innovations in 
teaching techniques, a small section of an electrical engineering course was 
selected. The class was composed of eight volunteer students from a larger 
class of about sixty students. They were informed that some informal experi- 
mentation in teaching methods would be tried, but the class would cover the 
same amount of material as usual. The instructor was a Ph.D. candidate on 
his first teaching assignment and very amenable to trying out various ap- 
proaches. 



From the earlier observations of other classes, it was felt that com- 
munication between teacher and student could be Improved. At the beginning 
of the quarter this was briefly discussed with the entire class. All agreed 
it would be desirable to work at improving communication. One approach was 
to reserve a few minutes at the end of each class session for student evalu- 
ation of the presentation of material and other classroom activity. These 
notes were given the Instructor at the end of each week. 

The instructor was asked to facilitate communication by organizing the 
presentation of his content in such a way that there would be natural stop- 
ping points. At such points he would pose a question about the material 
just presented and ask all members of the class to write their answers. After 
a few minutes for each student to think about the question, there would be a 
general discussion of the correct solution. 

From our work in other courses it became clear that "problem- solving 
strategy" was an inherent part of the curriculum. It was never emphasized, 
however, in a manner which made the student aware of the process. Problem- 
solving strategy seemed to be taken for granted or was imbedded in the con- 
tent of the course. Our previous observations suggested that the problem- 
solving methods of many engineering students were quite disorganized. Stu- 
dents rarely approached their typical problems with any definite plan of pro- 
cedure . 

It seemed desirable, therefore, to make the discussion of problem-solv- 
ing strategy an explicit goal of the course. Students were asked to discuss 
their procedure for solving particular problem sets. This statement included 
a time- allotment schedule: each student was to indicate the time spent in 

initial thinking about the problem. Including efforts which led to incorrect 
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After the first problem set was turned In, the instructor led a general 
discussion about problem-solving methods. The need for spending more time 
thinking before plunging in was repeatedly emphasized. Along with this, was 
emphasis upon having a clear plan for systematically attacking a problem. 

For each problem set turned in to the instructor, he would make comments about 
the student's problem-solving strategy. The instructor in his written comments 
would raise questions about why the student took one approach rather than an- 

Other. 

In an. effort to focus on these strategies in the classroom, the instruc- 
tor's questions were so framed as to Illustrate the general approach. For ex- 
ample, students were sometimes asked to anticipate the next step on the basis 
of previous information. In the presentation of a circuit system, for instance 
students would be requested to provide some of the appropriate components . 

Most of the students in the course felt it was very worthwhile, both in 
terms of what they learned and from the perspective of an interesting adven- 
ture in learning. The long-heard cry for more teacher-student interaction was 
satisfactorily met. Nevertheless, some students were uninterested and could 
not be drawn into the learning process. We mention this only to illustrate 
the point that some students, at a given point in their lives, will not be 

open to learning. 

of greater significance is the varying degree of dissatisfaction that 
students who did well in the course felt with its general philosophy. Here 
students were not complaining about how things were being done. We know that 
overall they were fairly well satisfied. Instead, they were raising questions 
about ^ they were being expected to learn this way in the first place. The 
large amount of material and necessary memorization that went with it were not 
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congruent with their wishes. 

We have been concerned with examining the effects of encouraging more 
student** teacher communication. Overall, this seems both desirable and bene* 
ficial to the learning process. It does seem to open Pandora's box, however, 

i 

because the students begin to raise fundamental questions about the aims of 
a given course and the goals of the undergraduate curriculum. 



Conclusions and Implications 

Teaching is one of the major functions of the faculty. The goals of 
these teaching efforts can be described at different levels of analysis. At 
one level the primary goal of the instructor is the presentation of the spe- 
cific content of the course. It has been the intent of this report to demon- 
strate that the methods used in presenting the content of the course and the 
methods used for evaluating a student's mastery of the course content have 
far reaching consequences. The full extent of these -consequences can only be 
understood if we move to another level of analysis. Here we have to examine 
the goals of the entire curriculum. Only then can we -examine what contribu- 
tion a given course is making toward that goal. 

It is generally agreed that mastery of the content of the curriculum is 
only part of the larger goal of the curriculum. The other parts are a little 
more elusive when we try to define them and even more elusive when we try to 
implement them. The following is one statement of these larger goals: "We 

must encourage our students to learn to think independently, evaluate their 
own and others' work with high critical standards and be free enough to see 
creative alternatives. We must also encourage in our students the motivation 
to continue learning and to actively pursue their own intellectual curiosity. 
We must also develop in our students an awareness of the conq>lexity of the 
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technological revolution that is occurring in our society and an awareness of 

their own individual role in such a society.” 

While such goals may be stated differently there is an underlying theme 
which would represent a consensus. Our expectations for a student, s develop- 
ment go well beyond the accumulation of facts or even the techniques for solv 
Ing complex yet well-defined problems. In light of these larger expectations 



we can examine the consequences of teaching in any given course and further- 
more examine the consequences of the current structure of the curriculum. In 
this study we have explored several questions which have relevance to these 
issues and have arrived at the following conclusions: 

For a narrow range of high ability students there is little or no cor- 
relation between academic performance and Intellectual aptitudes. Instead a 



for more of the variability in student's academic performance. There is some 



than creativity are highly correlated with success in the engineering curricu- 
lum. 

Although creativity and flexibility are recognized as Important in- 
gredients in the problem-solving orientation of engineers, there is little 
evidence of this being a major focus of attention in the training of engineers. 
Even where an Instructor may reveal some of these characteristics in his teach- 
ing, there is a lack of follow through when it comes to evaluating the student s 
performance. A wide variety of factors make the student more responsive to what 



he is being taught. This Inadvertently leads students to focus on learning in 
a passive manner rather than actively pursuing the intellectual challenges of 



0 variety of variables that are associated with motivation and attitude account 



evidence which suggests that attitudes having to do with perserverence rather 






the course material. 
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By instituting a variety of means for students and instructors to com- 
municate with each other, many of the larger goals of the curriculum can be 
made relevant to any subject. Evaluation of the learning process becomes 
something that both teacher and student engage in and thus share in a com- 
mon enterprise. 
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